1. Field of the Invention
The invention relates generally to clustered storage systems and more specifically is related to improved methods and structure for shipping I/O requests between storage controllers of the clustered storage system.
2. Related Patents
This patent application is related to the following commonly owned U.S. patent applications, all filed on the same date herewith and all of which are herein incorporated by reference:                U.S. patent application Ser. No. 13/432,131, entitled METHODS AND STRUCTURE FOR TASK MANAGEMENT IN STORAGE CONTROLLERS OF A CLUSTERED STORAGE SYSTEM;        U.S. patent application Ser. No. 13/432,213, entitled METHODS AND STRUCTURE FOR DIRECT PASS THROUGH OF SHIPPED REQUESTS IN FAST PATH CIRCUITS OF A STORAGE CONTROLLER IN A CLUSTERED STORAGE SYSTEM;        U.S. patent application Ser. No. 13/432,223, entitled METHODS AND STRUCTURE FOR LOAD BALANCING OF BACKGROUND TASKS BETWEEN STORAGE CONTROLLERS IN A CLUSTERED STORAGE ENVIRONMENT;        U.S. patent application Ser. No. 13/432,225, entitled METHODS AND STRUCTURE FOR TRANSFERRING OWNERSHIP OF A LOGICAL VOLUME BY TRANSFER OF NATIVE-FORMAT METADATA IN A CLUSTERED STORAGE ENVIRONMENT;        U.S. patent application Ser. No. 13/432,232, entitled METHODS AND STRUCTURE FOR IMPLEMENTING LOGICAL DEVICE CONSISTENCY IN A CLUSTERED STORAGE        U.S. patent application Ser. No. 13/432,220, entitled METHODS AND STRUCTURE FOR MANAGING VISIBILITY OF DEVICES IN A CLUSTERED STORAGE SYSTEM;        U.S. patent application Ser. No. 13/432,150, entitled METHODS AND STRUCTURE FOR IMPROVED BUFFER ALLOCATION IN A STORAGE CONTROLLER; and        U.S. patent application Ser. No. 13/432,138, entitled METHODS AND STRUCTURE FOR RESUMING BACKGROUND TASKS IN A CLUSTERED STORAGE ENVIRONMENT.        
3. Discussion of Related Art
In the field of data storage, customers demand highly resilient data storage systems that also exhibit fast recovery times for stored data. One type of storage system used to provide both of these characteristics is known as a clustered storage system.
A clustered storage system typically comprises a number of storage controllers, wherein each storage controller processes host Input/Output (I/O) requests directed to one or more logical volumes. The logical volumes reside on portions of one or more storage devices (e.g., hard disks) coupled with the storage controllers. Often, the logical volumes are configured as Redundant Array of Independent Disks (RAID) volumes in order to ensure an enhanced level of data integrity and/or performance.
A notable feature of clustered storage environments is that the storage controllers are capable of coordinating processing of host requests (e.g., by shipping I/O processing between each other) in order to enhance the performance of the storage environment. This includes intentionally transferring ownership of a logical volume from one storage controller to another. For example, a first storage controller may detect that it is currently undergoing a heavy processing load, and may assign ownership of a given logical volume to a second storage controller that has a smaller processing burden in order to increase overall speed of the clustered storage system. Other storage controllers may then update information identifying which storage controller presently owns each logical volume. Thus, when an I/O request is received at a storage controller that does not own the logical volume identified in the request, the storage controller may “ship” the request to the storage controller that presently owns the identified logical volume.
FIG. 1 is a block diagram illustrating an example of a prior art clustered storage system 150. Clustered storage system 150 is indicated by the dashed box, and includes storage controllers 120, switched fabric 130, and logical volumes 140. Note that a “clustered storage system” (as used herein) does not necessarily include host systems and associated functionality (e.g., hosts, application-layer services, operating systems, clustered computing nodes, etc.). However, storage controllers 120 and hosts 110 may be tightly integrated physically. For example, storage controllers 120 may comprise Host Bus Adapters (HBA's) coupled with a corresponding host 110 through a peripheral bus structure of host 110. According to FIG. 1, hosts 110 provide I/O requests to storage controllers 120 of clustered storage system 150. Storage controllers 120 are coupled via switched fabric 130 (e.g., a Serial Attached SCSI (SAS) fabric or any other suitable communication medium and protocol) for communication with each other and with a number of storage devices 142 on which logical volumes 140 are stored.
FIG. 2 is a block diagram illustrating another example of a prior art clustered storage system 250. In this example, clustered storage system 250 processes I/O requests from hosts 210 received via switched fabric 230. Storage controllers 220 are coupled for communication with storage devices 242 via switched fabric 235, which may be integral with or distinct from switched fabric 230. Storage devices 242 implement logical volumes 240. Many other configurations of hosts, storage controllers, switched fabric, and logical volumes are possible for clustered storage systems as a matter of design choice. Further, in many high reliability storage systems, all the depicted couplings may be duplicated for redundancy. Additionally, the interconnect fabrics may also be duplicated for redundancy.
While clustered storage systems provide a number of performance benefits over more traditional storage systems described above, the speed of a storage system still typically remains a bottleneck to the overall speed of a processing system utilizing the storage system.
In the clustered storage system environment, a host system may direct I/O requests to any logical volume of the clustered system. However, a host that is tightly coupled with a storage controller may direct all I/O requests only to that storage controller (i.e., where, as in the configuration of FIG. 1, the storage controller is integral with a single host system—e.g., an HBA in that host system). The storage controller must determine whether it or another storage controller in the system owns the logical volume to which the request is directed. Or, where a host is coupled through a switched fabric to all storage controllers of the system (as in the configuration of FIG. 1) a host may direct an I/O request to a controller that it understands is the owner of the logical volume. However, based on communications among the storage controllers, ownership of an addressed logical volume may have changed (e.g., for load balancing or as a result of a fail over of another storage controller). The host may have directed the I/O request to a storage controller that no longer owns the addressed logical volume if the host has not yet been notified of such a change of ownership.
In some prior techniques, the controller receiving the I/O request directed to a logical volume that it does not own processes the request to generate low level I/O requests to the affected physical storage devices. The low level I/O requests so generated may be performed by the controller (since all controllers are coupled with all storage devices in the clustered architecture). However, this requires complex coordination with another storage controller that presently owns the addressed logical volume. In other prior techniques, the controller receiving the I/O request would generate the lower level I/O operations directed to the affected physical storage devices but then ship those lower level physical device requests to another controller that owns the addressed logical volume. The other storage controller would process the lower level I/O in coordination with its management of the addressed logical volume that it owns. Where the original I/O request was for writing data to the logical volume, the first controller would receive the data from the requesting host and forward that data to the other controller as part of the lower level I/O operations shipped across. In the case of a read request, the other controller would perform the lower level read operations and return data to the first controller that, in turn, returned the requested data to the requesting host system. Or, in the case of a write request, the first controller (in receipt of the request directed to a logical volume) would receive the write data from the host system and forward that data to the second controller along with the lower level write operations. In other prior techniques, the controller receiving the I/O request would generate lower level operations only then to realize that the resources for those lower level operations were owned by another controller. Responsive to such a determination, the controller would simply discard the work that had been completed to decompose the logical volume request into corresponding lower level requests and ship the original logical volume request to the other controller (thus requiring duplication of the computational efforts to decompose the logical volume request on the other controller). These prior techniques result in significant processing in both the first controller that received the request and in the other controller that actually performs the required lower level I/O operations. Further, these prior techniques could “double buffer” the data associated with the original request by storing the related data locally and then forwarding the data to the intended recipient thus adding still further processing as well as memory requirements in the first controller.
Thus it is an ongoing challenge to process I/O requests in a storage controller of a clustered storage system by shipping aspects of the received request to another storage controller where ownership of logical volume may be transferred among the controllers.